Population heterogeneity in the fractional master equation, ensemble self-reinforcement, and strong memory effects

We formulate a fractional master equation in continuous time with random transition probabilities across the population of random walkers such that the effective underlying random walk exhibits ensemble self-reinforcement. The population heterogeneity generates a random walk with conditional transition probabilities that increase with the number of steps taken previously (self-reinforcement). Through this, we establish the connection between random walks with a heterogeneous ensemble and those with strong memory where the transition probability depends on the entire history of steps. We find the ensemble-averaged solution of the fractional master equation through subordination involving the fractional Poisson process counting the number of steps at a given time and the underlying discrete random walk with self-reinforcement. We also find the exact solution for the variance which exhibits superdiffusion even as the fractional exponent tends to 1.


I. INTRODUCTION
Anomalous diffusion appears in many natural processes in physics, chemistry and biology when measurements of mean squared displacement m (2) (t) show nonlinear dependence on time: m (2) (t) ∝ t µ [1][2][3][4][5].A variety of models has been suggested for anomalous diffusion including continuous time random walk [6], fractional Brownian motion [7], generalized Langevin equation [8][9][10] and Lévy walks [11,12].A typical feature of anomalous transport models involving temporal subdiffusion and superdiffusion is the appearance of memory effects.When a stochastic process depends on a series of previous events, it is often referred to as having non-Markovian characteristics or memory.In many natural phenomena, memory is a recurring theme, such as earthquakes [13], quantum physics [14][15][16] intracellular transport [17][18][19][20], and cell motility [21].Another direction to model anomalous diffusion is through random walks that account for the whole history of its past, described as strong memory [22][23][24][25][26][27].However, it is difficult to justify why natural processes should exhibit such strong memory effects as seen in elephant random walks [22], especially for inanimate objects such as intracellular organelles.In efficient search strategies [28] that have an essential role in time sensitive biological processes [29], strong memory has significant effcts [30].More recently, it was shown that strong memory and reinforcement can generate superdiffusion in a continuous time and finite velocity strong memory model [31], even in the presence of rests [32].However, when including a trapping state, the superdiffusion caused by reinforcement was only transient [33].
In biology, cell motility and intracellular transport often exhibit anomalous characteristics and memory effects [17,20,34,35].The movement of organelles is often subdiffusive due to the crowded cytoplasm [35], which is in direct contrast with the need to efficiently and quickly transport material to specific targets, accomplished by active transport.Apart from this, cellular populations are almost always heterogeneous [36], an example being the different molecular expression levels across individual cells in the brain [37][38][39].Furthermore, single cells contain ∼ 10 2 − 10 3 heterogeneous vesicles with various sizes, morphology and motion essential for all eukaryotic life such as lysosomes [40].Mathematically, models accounting for static population heterogeneity need to be explored so that the 'population-averaged assays' [36], which pervade much biological literature [19,35], can be accurately quantified and the effects of small yet important subpopulations properly identified [36].
The aim of this paper is to explore the effects of population heterogeneity, characterized by a distribution in transition probability, on the fractional master equation.Below, we demonstrate how heterogeneity changes the fundamental characteristic of the fractional master equation, used in modelling many biological processes that exhibit anomalous trapping [41,42].The effective underlying random walk exhibits self-reinforcement due to the ensemble averaged conditional transition rates increasing as previous steps accumulate.Moreover, by introducing heterogeneity into the fractional master equation generates ballistic superdiffusion even when the fractional exponent µ → 1.We show from a random walk perspective the reason behind why heterogeneity is needed in natural phenomena for efficient transport of an ensemble.Furthermore, we show the mathematical link between population heterogeneity and strong memory.While the topic of random walks in heterogeneous, random environments have been covered extensively in literature [43,44], it will not be treated in this paper.

II. FRACTIONAL MASTER EQUATION WITH RANDOM TRANSITION PROBABILITIES
The anomalous movement of particles on a lattice that experience trapping with heavy-tailed waiting times can be described by the fractional master equation [6] Here p is the probability to find the particle at position x = ka (k ∈ Z) and time t.The anomalous escape rate i(x, t) is defined as and D 1−µ t is the Riemann-Liouville derivative Equation ( 1) describes a random walk where a particle leaves its current state x at time t with rate i(x, t) and either jumps with constant probability q or 1−q to x+a or x−a, respectively [6].The anomalous rate defined in Eq.
(2) characterizes waiting times that are Mittag-Leffler distributed [45].From (1), by setting q = 1/2 and taking the continuous space limit, one can obtain the fractional diffusion equation ∂p/∂t Equation (1) with q = 1/2 and the fractional diffusion equation produces subdiffusive behavior characterized by the mean-squared displacement (and also the variance since the mean is zero) m (2) (t) ∼ t µ where 0 < µ < 1.In order to model population heterogeneity, q, the probability of jumping one step in the positive direction, now becomes a random variable for each independent realization of a random walk.In this case, what is the behavior of the ensemble average of the heterogeneous population?
Clearly for many biological processes, such as intracellular transport [17], the value of q is heterogeneous across the population of particles.Since the bias parameter q is related to the speed as v ∼ 2q − 1, therefore, q can be obtained from the speed distribution in experiments.Population heterogeneity in speeds is evident in many publications on the topic of intracellular transport [20,35,46] and cell motility [34].To account for the heterogeneity across a population of particles, consider that q in Eq. ( 1) is a random variable that is Beta distributed with a probability density function where B(α + , α − ) is the beta function.
If q becomes random, how does the anomalous behavior in Eq. ( 1) change?One might reasonably expect that ensemble fluctuations in q will increase the dispersion of particles leading to randomness of the fractional diffusion coefficient.This idea for standard diffusion has been considered by theories of 'diffusing diffusivity' [47][48][49] and such heterogeneity was demonstrated to be advantageous for biochemical processes triggered by first arrival [29].Moreover, heterogeneity can be modelled in many ways such as a non-constant diffusion coefficient [50][51][52] or a non-constant anomalous exponent [41,45,[53][54][55].Dichotomously alternating force fields in the fractional Fokker-Planck equation have also been used to model temporal heterogeneity [56].
In what follows, we will demonstrate that the randomness of q leads to the phenomenon of ensemble selfreinforcement and is also connected to random walks exhibiting strong memory.To show this, we need to find the explicit expression for the ensemble averaged probability p(x, t) in continuous time defined as where p(x, t|q) is the solution for the master equation ( 1) with a single value of q.In order to do this, we first consider the underlying discrete time random walk for (1) and then utilize the idea of subordination [6,57].

III. ENSEMBLE SELF-REINFORCEMENT AND STRONG MEMORY EFFECTS
The underlying discrete time random walk for Eq. ( 1) is described by the difference equation where the random jump ξ n = ±a with probability q and 1 − q respectively, and X 0 = 0.The conditional probability obeys the master equation The solution [57] is The particle reaches the point x at time n if it makes 1 2 (n+x/a) positive jumps and 1 2 (n−x/a) negative jumps.Next, we define a probability function which describes the effective underlying random walk for Xn such that By averaging (8) using f (q) from ( 4), we obtain the master equation where the transition probabilities u + n (x) and u − n (x) are defined as follows (13) Transition probabilities (13) follow from averaging (8) with respect to f (q).By using the solution (9) we find Surprisingly, randomness of the parameter q generates effective transition probabilities, u ± n (x), which describes the ensemble self-reinforcement phenomenon.It follows from ( 14) that the probability to step in the positive or negative direction increases as more steps in those directions are made in the past, which is known as selfreinforcement.In what follows, we demonstrate the link between Eqs. ( 12) with ( 14) and random walks with transition probabilities dependent on the entire history of its past, a property called strong memory.Furthermore, we provide an explanation to how these two concepts are linked despite the difference in the underlying mechanism.
In fact, Eq. ( 12) describes a random walk with strong memory: Xn+1 = Xn + ξn+1 .The conditional transition probability for the discrete steps, ξn , depends on its entire history such that Here n ± is the number of steps taken in the positive and negative directions, respectively.Equation ( 15) can be obtained from the transition probabilities ( 14) by combining the current position x = a(n + − n − ) and the total number of steps n = n + + n − .The transition probabilities (15) depend on the entire history because n ± counts the number of steps taken in the positive and negative directions up to time n.This dependence of the conditional transition probability on the entire history of the random walk is known in the literature as strong memory [22-27, 31, 32].The conditional transition probability ( 15) is exactly the same as that of a Pólya urn model [27,57] where initially the urn contains α + red and α − black balls and then only one ball is added per draw with n ± the number of red and black balls drawn, respectively.Comparing Eqs. ( 14) and (15), it is clear that ensemble self-reinforcement generates strong memory effects.However, a key feature of the random walk governed by (12) is that the strong memory effect is a by-product of the heterogeneity in the ensemble.Does this mean that, through heterogeneity, particles performing the random walk in (12) are somehow more likely to step in the positive or negative direction dependent on their history?On the contrary, this ensemble self-reinforcement is a consequence of sampling a heterogeneous population.This type of effect that leads to reinforcement is discussed in probability theory as aftereffect or spurious contagion [57].Rather than steps becoming more likely given the previous step, particles with a very high propensity to always step to the right or left are more likely to be found at the positive or negative extremities of the population.This is especially pertinent in cell biology as often in microscopic scales, such as intracellular organelles, there is no internal mechanism of reinforcement or 'contagion' and memory effects could be due to sampling a heterogeneous population.Equations ( 14) illustrate the fact that simply changing the transition probability q from a constant to a random variable completely changes the fundamental underlying mechanism of transitions in the ensemble.

IV. ENSEMBLE AVERAGED SOLUTION FOR THE FRACTIONAL MASTER EQUATION
By using the concept of subordination [6,57], we can find the explicit expression for the ensemble averaged probability distribution p(x, t) in continuous time defined in (5).The underlying random walk for the master equation (1) is the compound fractional Poisson process [58] X µ (t) = where ξ i are random jumps, N µ (t) is the fractional Poisson process and X µ (0) = 0.The latter describes the number of steps taken at time t given the waiting time is Mittag-Leffler distributed [58].Using subordination [6,57], we can write where P (x, n) is defined in (10) and Q µ (n, t) = Prob{N µ (t) = n}.One can also write down p(x, t) in terms of the position of the continuous-time random walk where From the master equation (12) or by averaging the solution (9) as shown in (10), one can obtain The probability Q µ (n, t) is given by [58], So substituting ( 20) and ( 21) into (17) gives the ensemble averaged solution of the fractional master equation (1) through subordination involving the fractional Poisson process and the underlying discrete random walk with self-reinforcement.Fig. 1 illustrates the solution (17) obtained by Monte Carlo simulations for the symmetrical case (α + = α − ).One can see the unusually strong dispersion for the subdiffusive master equation, which is a result of the interaction between ensemble self-reinforcement described by P (x, n) and heavy-tailed waiting times with divergent mean described by Q µ (n, t).

A. Monte Carlo simulations
The simulations for all figures were performed in the following way: 1. Initialize N particles at X(0) = 0.For each particle, the value of q is a random variable drawn from a Beta distribution.
3. Iterate until a required end time t end .

V. SUPERDIFFUSION GENERATED BY ENSEMBLE SELF-REINFORCEMENT
Now, we will show how ballistic superdiffusion can arise due to ensemble self-reinforcement.To do this, we need to find the moments corresponding to the discrete case of ( 17) Here, the summation is over all the lattice positions Ω = {ka} with k ∈ Z.Using (10), we can rewrite (22) as Recognizing that the summation in ( 23) is simply the m th moment of the discrete random walk, X n , governed by ( 8) for any fixed value of q, we find First, we find the conditional moments of the underlying random walk for fixed q: E(X n ) = G (1) and E(X n is the probability generating function [61].Performing this calculation, we obtain and The variance is proportional to n: Now, we take the average of ( 25) and ( 26) to obtain the variance of the effective random walk.In contrast to (27), the variance involves a term proportional to n 2 Var Xn =((2q where The difference between ( 27) and ( 28) is fundamentally important because the term proportional to n 2 generates ballistic superdiffusion

A. Symmetric Beta distribution: zero average advection
To avoid the averaged advection caused by an asymmetric Beta distribution, we only consider cases when the Beta distribution is symmetric, The absence of averaged advection is emphasised in Fig. 1, which shows symmetric distributions for different values of α. Figure 1 also shows that in the limit of large α, the distribution reverts back to the distribution typical for the subdiffusive regime.
For the symmetric case with q = 1/2, one can obtain The reason why the variance has a term proportional to n 2 can be explained by ensemble self-reinforcement expressed by the transition probabilities in (14), which leads to a greater dispersion of particles over time compared to standard random walks.Note that this result can be obtained by also finding the moments through a recursion relation from the master equation ( 12) [25].
One can find the variance for the effective continuoustime random walk where Xµ (t) is defined in (19), n 2 (t) and n(t) are derived from the fractional Poisson process [58] as and where Finally, the variance in continuous time is The appearance of superdiffusion is demonstrated by numerical simulations in Figs. 2 and 3. Figure 2 demonstrates numerically the relation in (36) and (37) since for values of µ < 0.5, Var Xµ (t) shows subdiffusion and for values µ > 0.5 shows superdiffusion.Moreover for µ = 0.5, Var Xµ (t) is exactly diffusive.Note that when µ = 1, N µ (t) becomes a Poisson process with rate 1/τ 0 and the variance becomes ballistic: This result is different from the case when an external force combines with the fractional master equation [62,63] where the first moment is m (1) (t) ∼ t µ and so the second moment becomes m (2) (t) ∼ t 2µ .The superdiffusion caused in this new process is a result of a heterogeneous population of particles and this generates ensemble self-reinforcement demonstrated by (14).A simple random walk with bias and fractional rates would be described by (1) where q is a constant.Explicitly, the mean position and variance of this random walk conditional on the transition probability are [62][63][64] E[X(t)|q] = a(2q − 1) Γ(µ + 1) Var [X(t)|q] =(2q − 1) Clearly, (39) exhibits superdiffusive behavior but the terms proportional to t 2µ disappear when µ = 1.The reason for this is that the underlying random walk model, X n , has variance proportional to n, as seen in (27).However, (36) exhibits ballistic superdiffusion when µ = 1 because the effective random walk of the ensemble, Xn , has variance (26) proportional to n 2 and n.
Furthermore, from this new, heterogeneous population model we are able to achieve a smooth transition in time between subdiffusion and superdiffusion.This is evident by increasing the value of α → ∞.This is intuitive as the symmetric Beta distribution approaches a delta function centered at q = 1/2 as α → ∞ and so we recover the standard fractional master equation and the resulting subdiffusion.However when α ∼ 1 and µ > 1/2, we obtain superdiffusion in the long-time limit.This transition between superdiffusion and subdiffusion is demonstrated using computational simulations in Fig. 3.

VI. DISCUSSION
Although there is vast literature on strong memory effects in statistical physics [22-27, 31, 32], many elephant random walk-like models lack the mechanism of how the strong memory is produced.Given that a heterogeneous population of random walkers emulates strong memory, this opens a new avenue for modelling biological processes that display strong memory properties and yet are heterogeneous ensembles of inanimate objects, like organelles and micromolecules.Might it be that nature has developed a mechanism like ensemble self-reinforcement that we demonstrate in (14) as a proxy for strong memory?Such questions have plagued the field of intracellular transport for decades where brainless membranebound vesicles seemingly engage in random walks that appear to have correlations caused by strong memory effects [18,20].For example, a high value of q might represent a higher affinity to attach to the dynein family of motor proteins and therefore the particle moves very directionally towards the cell nucleus whereas a low value of q would be a higher affinity to attach to kinesin which moves towards the cell periphery.A value of q ∼ 1/2 would imply that a particle may have equal chance to move in either direction.sThe relationship between q and speed v of a vesicle is v ∼ 2q − 1.So the bias parameter q can be obtained from experiments.Heterogeneity in velocities of intracellular vesicles is well-established [20,35,46].Ensemble self-reinforcement enables the organization of directional movement as an ensemble effect from heterogeneity.Furthermore, we showed that ensemble self-reinforcement can generate ballistic superdiffusion.
This finding also fits nicely with the emerging theory that, in biological processes, the first arrival times of a signal to a cell (or neuron) influence the subsequent system behavior far more than the average arrival times [65].With ensemble self-reinforcement the cell can organize the movement of these particles such that it maintains efficiency of transport and overcomes the trapping that occurs in the crowded cytoplasm.We hypothesise that ensemble self-reinforcement is a way that the cell efficiently transports vesicles in a heavily crowded intracellular environment, which has been shown to be subdiffusive [19,41].

VII. SUMMARY
In this paper, we formulate a fractional master equation with random transition probabilities across the populations of random walkers.This population heterogeneity generates ensemble average transition probabilities that that increase with the number of steps taken previously, which we call ensemble self-reinforcement.These averaged transition probabilities open a new avenue to model strong memory effects through a heterogeneous ensemble of random walkers.Furthermore, we show analytical solutions for the variance and probability density function of the ensemble averaged, effective random walk.
Through this, we establish the connection between random walks with a heterogeneous ensemble and those with strong memory where the transition probability depends on the entire history of steps.We find the ensemble averaged solution of the fractional master equation through subordination involving the fractional Poisson process counting the number of steps at a given time and the underlying discrete random walk with self-reinforcement.We also find the exact solution for the variance which exhibits superdiffusion even as the fractional exponent tends to 1.This paper demonstrates that heterogeneous populations of anomalous random walks can achieve effective transition probabilities describing strong memory, which we call ensemble self-reinforcement.We find that such heterogeneous populations overcomes heavy-tailed waiting times with divergent mean to exhibit ensemble superdiffusion thus revealing an intrinsic advantage of heterogeneity.Moreover, this provides a new mechanism through which seemingly unintelligent systems can exhibit strong memory.